Skip to main content

Current Data Import Processes

warning

For reference only. This process will occur outside of I2C's scope. This is included here for reference for the AANP team.

Summary​

This document outlines the CURRENT proccess used to manage data imports from the National Provider Identifier (NPI) and State Board of Nursing (SBON) datasets into the organization's CRM system. The primary goal is to understand the current Nurse Practitioner (NP) population in the United States and use comparative data from previous years to identify trends.

Introduction​

The organization imports data from the NPI and SBON sources to identify Nurse Practitioners (NPs) and update existing records. This process is critical for maintaining accurate and up-to-date member information, supporting strategic planning, research, and various organizational functions including membership management, marketing, and compliance.

If an awesomely perfect match is found, the system will automatically link the NPI record to the existing profile in the CRM system. If a possible match is identified, the system will flag it for manual review by internal users. The matching process involves comparing multiple data points such as name, address, license number, and NPI number to determine the likelihood of a match.

If after manual review, no match is found, the user marks the record as a non-match, and no new record is created in the CRM system. This is where issues begin to arise, as the system may not be able to identify all possible matches due to variations in data quality, formatting, or missing information. Duplicates are a common issue that can lead to data inconsistency, errors, and inefficiencies in data management.

If a match is found, the following objects are created or updated in the CRM system:

Persons > Board State Address​

  • Board State
  • Board Year
  • Date Processed
  • Board Address
  • Company Name
  • Line 1
  • Line 2
  • City
  • State/Province
  • Postal Code
  • Locality Name
  • Country

If no match is found, then a new Person record is created in the CRM system with the following attributes:

Person​

  • FirstName
  • MiddleName
  • LastName
  • Phone
  • Email1
  • Member Type: Individual Non-Member

Persons > Board State Address​

  • Board State
  • Board Year
  • Date Processed
  • Board Address
  • Company Name
  • Line 1
  • Line 2
  • City
  • State/Province
  • Postal Code
  • Locality Name
  • Country

In both scenarios, if the Person does not have a valid address on file, the board state address gets copied over as the "Default Board State Address" for the Person record and used as their preferred address. This is a completely separate fieldset from the Board Address. The address is duplicated in two places.

Business Requirements​

External Matching Process​

  1. As an internal user, I should be able to prepare and clean the data before importing it for matching.
  2. As an internal user, I should be able to run initial matching algorithms to identify perfect and possible matches.
  3. As an internal user, I should be able to score possible matches based on a defined scoring system.
  4. As an internal user, I should be able to review high-score matches automatically and queue medium-score matches for manual review.
  5. As an internal user, I should be able to flag low-score matches for further investigation without importing them.
  6. As an internal user, I should be able to use batch review and automation tools to streamline the review process for medium-score matches.
  7. As an internal user, I should be able to iteratively refine matching rules and criteria based on patterns and results over time.

Data Import into CRM System​

  1. As an internal user, I should be able to import only the matched records into the CRM system, including essential fields such as Board State, License Number, NPI Number, and Year.
  2. As an internal user, I should ensure that no new records are created for unmatched items.
  3. As an internal user, I should be able to validate and verify the imported data to ensure accuracy.
  4. As an internal user, I should be able to update or add fields to identify that a contact has an NPI or State Board entry for membership access purposes.
  5. As an internal user, I should be able to automatically match records by NPI number.
  6. As an internal user, I should be able to flag potential matches requiring review and leverage multiple data points to help resolve difficult cases.
  7. As an internal user, I should be able to link matched NPI records to existing profiles accurately in the system based on the match results.

Records Processing and Validation​

  1. As an internal user, I should be able to review the matched records imported into the CRM system.
  2. As an internal user, I should be able to run scripts to remove standardized duplication and add NPI numbers.
  3. As an internal user, I should be able to hand review possible duplicates identified during the import process.
  4. As an internal user, I should be able to create a table with the full count of licensed NPs and the de-duplicated NP population.
  5. As an internal user, I should be able to analyze and report on linked NPI data separately from state board data for required metrics and monitoring.

Geocoding and Product Creation​

  1. As an internal user, I should be able to submit data for geocoding to get latitude and longitude information.
  2. As an internal user, I should be able to save the geocoded files in the appropriate folders for mapping.
  3. As an internal user, I should be able to use NP Count data to create various products such as Market Share Reports, County Level Maps, Congressional District Maps, GA Fact Sheets, Multi-License Analysis, and Workforce Survey Sampling Frames.

Closeout and Data Export​

  1. As an internal user, I should be able to export completed data from the staging area and from the Persons with Board Info view.
  2. As an internal user, I should be able to compare completed data with raw data to identify non-NP observations to be removed in the next data acquisition cycle.
  3. As an internal user, I should be able to review all administrative data and documents for completeness.
  4. As an internal user, I should be able to identify lessons learned from the data import process.

Assumptions​

  • All Internal Users will be included in the Personas definition.
  • Timelines will follow as outlined in each requirement.

References​

This document outlines the requirements for an efficient and compliant data import process that minimizes duplicate records and ensures high data quality.

Approvals / Signoffs​

RoleNameSignatureDate
Technical LeadJeff Sikes
StakeholderKate Bradley